Very Deep Multilingual Convolutional Neural Networks for LVCSR

摘要

Convolutional neural networks (CNNs) are a standard component of many currentstate-of-the-art Large Vocabulary Continuous Speech Recognition (LVCSR)systems. However, CNNs in LVCSR have not kept pace with recent advances inother domains where deeper neural networks provide superior performance. Inthis paper we propose a number of architectural advances in CNNs for LVCSR.First, we introduce a very deep convolutional network architecture with up to14 weight layers. There are multiple convolutional layers before each poolinglayer, with small 3x3 kernels, inspired by the VGG Imagenet 2014 architecture.Then, we introduce multilingual CNNs with multiple untied layers. Finally, weintroduce multi-scale input features aimed at exploiting more context atnegligible computational cost. We evaluate the improvements first on a Babeltask for low resource speech recognition, obtaining an absolute 5.77% WERimprovement over the baseline PLP DNN by training our CNN on the combined dataof six different languages. We then evaluate the very deep CNNs on the Hub5'00benchmark (using the 262 hours of SWB-1 training data) achieving a word errorrate of 11.8% after cross-entropy training, a 1.4% WER improvement (10.6%relative) over the best published CNN result so far.